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Change History 



1,1 Release Change Nummary 

Version 0.9 

• Worked/changed several of the oustanding issues. 
. . Added ct_sec_get_sec_mech routine to the MALI formal spicification 



Version 0.5 



Added the formal specification of the MPM Interface. This part does not affect tile CtS 
client that program to the MAL Interface. 



Version 0.3 



Combined Generic Authentication and ACL Management documents together. Rename 
the design documents. The new documents arc/afs/aix/project/chister/design/security/ 
cts_design.ps and .pdf. 

Added the formal specification of the MAL Interface. 



Version 0.2 



Updated document with the comments made at review. Better defined the situations 
affected by the change in security mechanism, and defined the steps during a mechanism 
migration. 

Updated some of the MALI and MPMI routines. 
Updated the security token descriptor. 
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Change History 




• Initial proposal for the cluster security services generic authentication interface. 1 
interface is based on the SP's S3 interface. This document also includes the portin; 
S3 interface from SP to cluster. Some of the routines were deprecated and new ro 
were added. 

1.2 14ms Jtem Tracing information 

The following CMVC line item tracking feature covers this line item: 49095 
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Introduction to Cluster Security Services 



2.0 Introduction to Cluster Security Services 



Ije a 
pro- 
setbure 



The cluster system, being a collection of nodes that are tied together by a network, must provid 
secure way of sending messages between a client and a server application. The security services 
vides a set of routines that allows an application to acquire network identity and to establish 
communication with its clients or other applications in the cluster. It also offers a means to enforce 
access control to resources based on network identities. The interface provided by security 
generic 1 and hides the complexity of using the underlying security mechanism and differences 
between different mechanisms. The ultimate configuration and administration of the underlying; 
rity mechanism is still required, however, this is not part of this component design document. 



servi ces is 



secu- 



is to 
exist- 



tiat 



24 ^story 

The SP security services provide the S3 library and interface, which at the moment are implemented 
using DCE. This is the only mechanism supported by the S3 interface, and basically, it is a trie idly 
wrap around DCE routines customized for the SP environment. The long term goal in the cluster 
eliminate the dependency on DCE and at the same time to keep the S3 interface as close to the 
ing one as possible. Because some of the S3 routines export DCE data structures and DCE specific 
mechanisms (e.g. ACL management, key management), they will be removed. Other routines 
deal with DCE groups will be deprecated, although supported for backwards compatibility. We 
courage the cluster services and cluster specific daemons to use die deprecated S3 routines 
routines specific to the SP configuration will be deprecated. There are major differences 
cluster and the SP, and new routines must be added to deal with this differences. Changes to thi 
authentication interface may occur while the installation and configuration component is being 
designed. 

The new S3 interface is divided into two parts: 1) generic authentication routines and 2) securiiy 
mechanism independent ACL management routines. Both parts are covered in this design document 



dis- 
, the 
betwetkn the 



. Also. 



2,2 Objectives 

The Cluster security services (CtS) interface must fulfill the following requirements: 

SECURITY: CtS must provide a secure environment for the cluster users. The level of 
security and data privacy/integrity depends on the underlying security 
mechanism in use. 



1 . For backwards compatibility with the S3 library in SP, CtS API contains several routines that work only with pCE as 
underlying security mechanism. These routines should not be used by cluster applications. 
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Introduction to Cluster Security Services 



PERFORMANCE: the CtS interface must provide good performance. The perforr lance 
will be influenced to a certain degree by the underlying authenticate c 
mechanism. 

RELIABILITY: the CtS interface must meet the reliability standards of the clustet. Its 
reliability depends to a certain extend on the underlying security mecha- 
nism in use. 

SCALABILITY: CtS must meet the scaleability standards of the cluster. Most of ihe 



underlying security mechanisms provide enough scaleability to meet 
standards of the cluster. 



EXPANDABILITY: CtS must provide an interface that will allow future security mech- 
anisms to be used. The API must be generic and at the same time it s lould 
provide the functionality required by the cluster operations and utilization. 



2.3 Overview 

The Cluster Security Services subsystem provides infrastructure elements that client-server applica- 
tions can use to implement authentication and access control. It is structured in multiple layers 
components, and offers a single set of routines, regardless of the underlying security mechanism that 
is being used. Figure 1 depicts a schematic representation of the CtS stracturing. 



The CtS subsystem is made of the following parts: 

1 Mechanism abstract layer (MAL) provides a generic interface to the security client|s. It 
exports only cluster and GSS specific data types and mechanisms. 

2 Mechanism plugable module (MPM) provides a mechanism specific implementation of 
the generic interface routines. All MPMs export a predefined set of routines. 

3 Compatibility module (CM) provides the routines that are part of the SP's security ser- 
vices and were deprecated in cluster. It is part of MAL and is not represented in the fig- 
ure. 

Note: Cluster applications should not use any routine in this module. These routinles are 
supported only for backwards compatibility. In the future, support for these rou- 
tines may be dropped. 

Two mechanisms are supported in the first release: Krb5 and Vanilla AIX. While Krb5 offers a high 
level of data privacy and integrity, the trust level for the Vanilla AIX mechanism is minimal antjl no 
data privacy /integrity service is being offered (TBD). 
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Figure 1 



Cluster Security Services structure 



Regardless of the security mechanism configured and used in the cluster, CtS provides ACL miuiage- 
ment support. The ACL management is mechanism idependent and limited to the registry and i " 
RMC subsystem. Figure 2 represents a graphical description of the relationship between i " " 
ments during authentication and authorization. 



ihe 

l different ele- 



|an 
con- 



A client uses its credentials to initiate a security context with a server (in our case a registry or 
RMC server). Once the server accepts the security context initiated by the client, it can use this 
text to get information about the client This information consist of the name of the client, wha: 
groups, if any, the client belongs to (the method of geting this information may be different fro n one 
security mechanism to another), etc. and it is part of the client's privilege attribute certificate (IfAC). 

If the client wants to access a resource managed by the server, then the server can use the clients 
PAC, the resourced ACL, and the permission set required for the operation requested by the client to 
determine if the client has suficient authority to access the resource. 
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Cluster Service (RMC or Registry) 



Required Perm. Set 
For Operation 



Access 
Control 




Figure 2 



Authentication and authorization in the cluster. 



This documents details the generic authentication and ACL management routines of the 
abstract layer, the compatibility module, and the mechanism plugable module's interface 

The following is a list of items that must be completed for this design: 



mechanism 



Table 1 Generic Authentication Interface Check Points 


Check Point 


Date 


Sketch of MPM interface (MPMI) and how the CtS interface (MALI) 
maps to MPMI. 


June 7 


Definition and manipulation of the opaque authenticated id (involves 
prototyping). 


June 28 


Formal specification of MALL 


July 12 


Formal specification of MPMI. 


July 19 
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Table 1 Generic Authentication Interface Check Points 


Check Point 


Date 


Pull in generic cluster discussion from ACL management design docu- 
ment. 


July 26 


Table 2 CtS Interface Check Points 


Checkpoint 


Date 


Pull out generic cluster discussion. 


July : 26 


Finalization of the interface specification (return codes). 


August 4 


Revised cluster security design document 


Sept. 15 


Stub library for the CtS interface 


Sept. 30 



2.4 NLS 



i message 



Being part of the cluster project, CtS is NLS enabled. As part of this support, CtS ships a 
alog, ct_sec.cat, and a message header file, ct_sec_msg.h. During each call to ct_sec _get_error_ 
routine, CtS tries to open the message catalog. If successful, tries to retrieve the message. 

The maximum length of the messages is limited to 1024 characters. 

The message catalog shipped by CtS contains error messages for the return codes generated by 
MALI routines. Each MPM will provide its own message catalog for its return codes 



2.5 Packaging 

The cluster security software is packaged in a separate lpp from the cluster code. Coresquisites must 
be created between the security lpp and the cluster client lpp. Also, die cluster base lpp will prerpq the 
security lpp. 

TBD: need to discuss w/Sammuel Benjamin in more detail. 
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3.2 Cluster Environment 

This chapter presents the security services environment and how they fit into the cluster picture. From 
the previous security discussions you should be familiar with concepts like network credential (iden- 
tity) and security context, and why applications must acquire and use them (steps required to establish 
a secure communication will be described later in the document). But in order to acquire and use a 
network identity, applications must be able to determine several things, like what is the underlying 
security mechanism used in the cluster; or wh^t identity are they going to use to acquire credentials: 
or what happens if a cluster system administrator changes the underlying security mechanism. 

All these questions must find an answer in order to create a reliable system that can dynamica ly be 
changed to the administrator's preference. The intension of CtS is to provide an interface that is 
generic, meaning that clients of this interface do not deal directly with the complexity of and t le dif- 
ferences between different security mechanisms. Clients program to only one interface and th<y are 
not impacted by the addition, in the future, of a new security mechanism to the cluster. The installa- 
tion and configuration of the new underlying mechanism must follow strict guidance rules (to come in 
the future) for the cluster to work correctly. 

3.2.1 Storing Configuration Information 

The Cluster Security system design document specifies a single security mechanism in use at h time 
in the cluster. One can regard this specification as a restriction, however, a good one. CtS can deter- 
mine, during initialization, what security mechanism is being in use, and can load and initialize the 
corresponding plugable module. Please read Section 5.0 Implementation of Mechanism Pluga 3le 
Modules (MPMs) for more details on the design of the plugable modules. 

To help CtS determine what security mechanism is configured and used in a cluster, this infon nation 
must be stored in the System Registry. We can define a table in the System Registry that contai as info 
about the underlying security mechanism: what mechanism, the time of configuration, the admin who 
configured it, -and some other info that may be required/helpful. 

Any user in the cluster (authenticated or not) should be allowed to read this information by cal ling an 
API routine and from the command line—we can offer a command that does that, for example, 
lssecinfo, with some flags...) and only the cluster administrator(s) have permission to change t lis 
information. The access control must be enforced by the System Registry server. 

Issue 1: Mechanism info table; structure and initialization. 

Need to determine what type of information is required in the mechanism table and 
how many entry we have to allow (taking into consideration that we suppoit more 
than one security mechanism during migrations.) 

Issue 2: Recovery from System Registry/security mechanism failures. 
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What happens when the System Registry is not functional? We, definitely, won't 
be able to retrieve security mechanism info. I guess if the System Registry is not 
functional, the entire cluster is not functional, and we have a bigger problem than 
reading the security mechanism info 

What if the System Registry is not functional because of a problem in the ur deriv- 
ing security mechanism? This can be solved by starting the registry server in a ser- 
vice mode that doesn't use CtS. It is up to the System Registry team to determine if 
this is a viable idea and, if it is, to define the functionality in this mode 

Three requirements will be filed with the System Registry design component (SR_design) for t le fol- 
lowing: 

1 start the registry without security enabled 

2 use of Unix Domain Socket for registry communication 

3 replication layer override for manual intervention 

Another piece of information that must be stored is related to the MPMs. When a new plug-in 
installed on a system, we must write some information about the plug-in (the name of the plug- n and 
the mechanism it corresponds to) in a local host configuration file, for example, /var/ct/<clustei_id>/ 
security/ct_sec.cfg. This file should be used by CtS only. Everybody will be able to read this file and 
only root will be able to write to it 

3.2.2 Changing Security Configuration 

The design of CtS provides flexibility to the cluster administrator in establishing theunderlyinjk secu 
rity mechanism to be used, so long as there is a plugable module for that mechanism. The plug -in 
mechanism is introduced to allow CtS to dynamically load the module required for a security Eiecba- 
nism. The consequence is that the cluster administrators) should be able to install and configure a 
new security mechanism, change the underlying security mechanism info in the Systbm Registry, and 
the cluster should continue to work correctly. There are some issues related to the change of tho secu- j 
rity mechanism, and some will not be answered in this design document (at least noticompletely), but 
in the install/config design document However, CtS should provide the correct functionality to] deal 
with these issues. 

Let's assume we have a client-server application that uses the CtS API. Both the client and the server 
have network credentials for a configured security mechanism, and they established la security con 
text. They are, indirectly, using the mechanism^ libraries and runtime. Now, supposejan administrator 
changes the configuration for the mechanism in use to a new security mechanism. We assume, for 
simplicity, that the new mechanism is functional and configured correctly (i.e. all the (required files are 
installed and all the principals/users have been created, etc.). 

Issue 3: Detect changes in the configuration of security mechanism. 
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CtS 



The question is at what point in time are the client and server supposed to dejtect 
change in the security configuration? A sane answer would be to ^ait until 
ents fail, at which point in time CtS could check if a change occuired, and if 
acquire network credential for the new security mechanism. The problem is 
the old security mechanism is still functional, the CtS client won't fail. 



a 

cli- 
so, 

hat if 



Let's take a more in-depth look at what is going on during a configuration change, Wd let's anajlyze 
the situations most affected by changes in security mechanisms. 

5.ZZ 1 Migrating To A New Security Mechanism \ 
Under normal circumstances, cluster trusted services (which are CtS clients) have network credentials 
and they are able to accept/initiate security contexts from their clients/with other services. Of course, 
for this to happen, the underlying security mechanism must be correctly configured dnd in use ([librar- 
ies and runtime). 

CtS clients have one security token (that references the security services environmerit info: network 
credentials, network name, etc.) and several context tokens (that reference the security context nfo: 
client's network identity, client's PAC, etc,). These tokens point to opaque blobs, and CtS clients don't 
care what these blobs contain. From the CtS's point of view, the blobs are made of tko parts: rr echa 
nism abstract information (the MAL level); and mechanism specific information (th^ MPM lev ;l) 
The latter is opaque to the MAL routines and strongly dependent on the security mechanism. This 
part is the heart of CtS and it is the most affected by a change in the security mechaiiism. 

Usne 4: What happens to established tokens when a configuration change\occurs? 

The security service token should not be affected by a change in sjecurity melcha- 
nism because it deals with mechanism abstract information. The tbken descriptor 
contains a list of mechanism specific data, and some members of this list ma[y be 
affected. 

The context tokens will definitely be affected because they are seciirity mechanism 
dependent 

i 

The configuration changes may have different effects, depending on how the change is performed 
Security mechanism changes are not that often (however, CtS must handle them), and I regard ihem 
as migrations. A first condition for a successful migration is that the new security meqhanism must be 
correctly installed and configured (I include here all the cluster specific configuration: creation bf 
principals/users, keys/passwords, local files, etc.) 

CtS intends to handle migrations in a seamless fashion, with certain limits, of coursej These limits are 
imposed by the unconfiguration/removal of the old security mechanism. A migration from one secu- 
rity mechanism to another must be designed and implemented with care for the datajfiles that a re 
already in use by the running CtS clients (otherwise unexpected results may occur, like memory vio- 
lations, illegal instructions, etc.). As long as the old security mechanism is still functional, CtS clients 
are able to use the security services correctly. ; 
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The problem is that we don't want the CtS clients to continue using the old security ijiechanism» how- 
ever functional it may be (the purpose of a migration is to use the new security mechknism). We want 
to detect a change in the CtS environment as soon as possible and take appropriate actions. 



3. 2. Z 2 Detecting Security Mechanism Changes i 
The security tokens represent long-lived data. They should be created at program inijtialization 
destroyed when the program finishes. In contrast, the context tokens are short-lived <jlata. They 
created when a request is coming from a client and destroyed when the client closes |the conneqtion. 



and 
are 



The logical points of checking for changes in the security mechanism are before initiating security 
contexts (for the client) and before accepting security contexts (for the server). With pther word s, CtS 
checks for changes when the clients call ct_sec_get_client_ creds ( ) to acquire the pro- 
cess *s credentials and initiate a security context with a server. 

Note: A CtS client can be a short-lived process (like a CLI command) or ajlong-lived 
cess (a cluster service's daemon). The first one depends on the network identity 
credential inherited from the invoker. The latter must do a : 
ct_sec_login_as__service ( ) in order to acquire network identity and creden- 
tial. Both may need to initiate security contexts with servers or other) services, 
security token descriptor contains information to determine in what Category the 
caller falls. 



pro- 
and 



Tie 



From the security token descriptor we can determine if the caller logged on to the network as a 
vice, therefore being a daemon. In this case, CtS checks if the configured security mjechanism < 
the System Registry) is the same with what it is in the security token descriptor. If itj is the sam 5, 
continues with initiating the security context. If it is a different security mechanism, j CtS does 
lowing: j 

. loads and initializes the plug-in for the new security mechanism. If it fajls, returns 
. determines the network name of the service for the new security mechanism (the neftwork 

name may be different from one security mechanism to another). ; 
. acquires network credentials for the daemon using the new security mechanism. If i 

returns error. 

. initializes a security context with the target service. If it fails, returns error. 



In case of a server (service daemon), CtS checks for changes in security mechanism! when it calls 
ct_sec_authenticate_client ( ) . The server calls this routine when it neeqs to process the 
client's request to accept a security context The client sends a context control data (CCD) buffer to 
the server, and the server passes it to this routine. We afld the mechanism code to th^ CCD buffer to 
make it easier for CtS to determine if the client and server share the same mechanisii. If the mecha- 
nism from the CCD buffer is the same with the security mechanism used by the server, then w<: are 
lucky. If not, CtS checks if the mechanism coming from client is the same with the c|ne in the System 
Registry. If it isn't, returns error. If it is, CtS does the following: 
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: fails, 
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• loads and initializes the plug-in for the new security mechanism. If it Ifails, returns erroi). 
» acquires network credentials for the new' mechanism. 

• processes the client's request to accept the security context. 

The description above of what is happening during a Emigration does not discuss sjinchroniza ion. Ttje 
security token descriptor may be used by seyeral threads to create context token descriptors for con ■ 
nections coming from different clients at the same tiihe. If one client is using the iew security mechja- 
nism, it doesn't mean that all the others are using it. CtS must provide a single write/multipb read 
type of synchronization to the data that is affected b) r changes in security mechanism configuration 
(most probably the mechanism specific info only). 

3.2.2.3 Handling Security Mechanism Errors j 

The design of CtS provides support for multiple security mechanisms at a time. This does n<j>t mean 
that we implement a negotiation protocol between a blient and a server to determine what security 
mechanism will be used. The support of multiple sedulity mechanism is only for inigration purposejs 



. When CtS acquires a new network identity and credential (for the new security mechanism), 
are that there are some security contexts for the old mechanism. They are still gooid so long 
security mechanism is functional. Problems occurs when some other actions are tajken, like v 
mission, or whatever operation that involves the security token (which has credenjtial for the 
mechanism). Whenever an MPM routine returns a security mechanism error, CtS jwill check 
are any changes in the security mechanism, and they; will be reported back to the paller. In 
context token related error, the CtS client must establish a new connection with the server, u 
new network identity and credential 



3.2.2,4 Unpredictable Behavior 
Until now we talked about migration under the assumption that the administrator keeps the (|>ld seed- 
rity mechanism functional for a while (at least until all security contexts expires). What happ ?ns if tl e 
administrator unconfigures and removes the old security mechanism? CtS clients rhay simply crash or 
hang, leaving the cluster in a unusable state. I guess the best answer is "the behavibr is unpredictably 
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3.5 Daemon to Daemon Communication 

Daemon to daemon communication refers to the communication between two instances of the same 
service. In cluster, all instances of a service share the same network identity and pafcsword/key. Each 
instance acquires network identity during process initialization which is constantly fenewed using the 
password/key. The synchronization of the keys in cluster is performed by the Key Jilanagement com- 
ponent of cluster security. 

Because all instances of a daemon have the same network indentity, the communication between 
them can be simplified. There are two approaches: (1) authentication-based communication; fi|nd (2) 
non-authentication-based communication, Let's discuss the two approaches in more detiaL 

3.5.1 Authentication-Based Communication 

This approach is characterized by the fact that an instance of a daemon (initiator) tikat needs tab com- 
municate with one or more instances of the same daemon (acceptors) must authenticate to only one 
other instance. The service ticket that the initiating instance obtains from the security mechan ism is 
valid for all the accepting instances because they share the same network identity. The initiatiig 
instance sends the service ticket to the other instances, authenticating to them. If mutual authentica 
tion is required, each accepting instance will send back a context control buffer (CCD). The initiator 
must authenticate only the first accepting instance and compare this CCD buffer with the CCDIbuffers 
received from the rest of die accepting instances. 

After authentication, the same session key is shared between the initiating instanceiand the accepting 
instances, and , therefore, encrypted/signed messages can be broadcasted 

3.5.2 Non-Authentication-Based Communication 

This approach differs from the previous one because the initiating instance does ndt authenticjates to 
the accepting instances. All instances already have a network identity and they shate the same key. 
The initiating instance can vise the shared key to encrytp/sign messages that are brdadcasted to the 
accepting instances. Each accepting instance can then decrypt/verify the message received, an& if sue 
cessfiil, it can guarantee that it was received from somebody that knows the key they share. 
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This approach implies new routines that CtS must provide to encrypt/decrypt and sign/verify mes- 
sages using a key versus a security context. 

Issue 8: What daemon-to-daemon communication approach should be used in the c luster? 
The correct answer to this issue is that tcp based daemons should use the authenti- 
cation-based communication approach for authenticating different instances, and 
the udp based daemon should use the non- authentication communication 
approach. 
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